Modern web development is compiled, transpiled, interpreted and virtualized. It’s a lot of abstraction layers to get a modern website built. Each of those layers is a place where something can go wrong. And the other day I ran into one of those "build layer" issues and struck out big trying to find a solution on the web. The problem was with webpack compiling with cache objects, specifically old cache objects that I’m not trying to use anymore but webpack don’t care so it includes them anyway cause the cache is telling it to. While I didn’t figure out the root problem, I did figure out a workaround.

A bit of background. I just rebuilt this website (a regular thing for me). And the CSS and Javascript are built together with Webpack, which is great, except when it’s not. I was customizing a premium BootStrap theme and being a general purpose theme it had way too much Javascript and CSS for a simple blog like this one (like a 5.5MB JS file by default). Needless bloat is bad, so I went through the exercise of pruning out things I didn’t need. So starting out the theme has a source file structure like this

├── src  
│  ├── fonts  
│  ├── img  
│  ├── js  
│  │  ├── app.js   
│  │  ├── modules  
│  │  └── settings
│  └── scss  
└── webpack.config.js

In the app.js file is a bunch of import statements that pull in the specific Javascript dependencies and their implementations. To remove these I was just commenting out the import "./modeules/jswhizbang"; lines one by one as I made sure I didn’t need them. So 5.5 megs of Javaascript down to a couple hundred kilobytes, perfect. I was moving on to the styling when the problems started.

The CSS is SASS based, and built/bundled together with the Javascript by webpack , so while I was still working on my styles I noticed a problem. The main derived app.js file that I got down to ~230KB was being rebuilt as a ~3MB monster again while I was updating the CSS styles. The problem was in the build cache. For some reason, webpack was using a cache file from somewhere halfway through my JS pruning exercise. I could make small modifications to the app.js source file and get it to build correctly. But if I switched to just working in CSS and not touching the app.js source the build would jump back to the old cache file and build this relatively massive Javascript file.

Researching this bug on the internet was fruitless, almost every page pointed to making sure your resources were included in your webpage with a web cache busting string in the filename. Even StackOverflow questions about the build cache were answered with web cache answers. So I started hunting for the cache file locally.

Luckily webpack gives you some details of how it’s putting things together:

[hardsource:5c40b140] Using 11 MB of disk space.
[hardsource:5c40b140] Tracking node dependencies with: package-lock.json.
[hardsource:5c40b140] Reading from cache 5c40b140...

The word "hardsource" and the string "5c40b140" are the clues here. The cache filename is a hash and webpack gives you the first eight characters of the hash as a unique identifier (a common convention with hashes). The word "hardsource" is a bit misleading as we’ll soon see, but the hash fragment is perfect. I mainly use Linux so the command line tool to use to find something is of course find.

$ find . -iname "5c40b140*"
./node_modules/.cache/hard-source/5c40b140ced80212dbf4baf2d211ef8aa247b5d1ae20bbffd94f091cf3b56111

Now an important thing to understand about caches is that they should never be required, correctly built they are always optional and if they are missing or fail everything should work fine without them (just slower). So the fix is just to nuke the cache.

$ rm -rf node_modules/.cache/hard-source/*

NOTE: Always double and triple check usage of rm -rf as a matter of professional discipline.

So deleting the cache files was all that was needed to fix the Javascript build. It would be nice if the webpack project could include functionality for cache clearing. I mean really, if you are going to use caching it’s really pretty important to also include some tooling to help manually manage that caching. But for now this is an easy workaround and hopefully writing about it helps somebody. It’s worth saying that you should pay attention to your webpack output or you might miss a problem like this. The watch for file changes and rebuild feature is great but it tends to make us not pay attention to it as long as everything is working.